Papers with lifelong model editing
LEMoE: Advanced Mixture of Experts Adaptor for Lifelong Model Editing of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for single and batch model editing fail to apply or perform sub-optimally when faced with lifelong model editing. |
| Approach: | They propose an advanced Mixture of Experts (MoE) adaptor for lifelong model editing that incorporates a novel KV anchor routing method to enhance routing consistency between training and inference stage. |
| Outcome: | The proposed method surpasses existing models while maintaining outstanding performance in batch editing task. |
Knowledge Decoupling via Orthogonal Projection for Lifelong Editing of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for enhancing large language models (LLMs) have achieved some success, but their knowledge understanding and memory capacity significantly degrades after extensive editing. |
| Approach: | They propose a method that stores the basis vectors of the representation space of past edits in a knowledge cache and projects the gradient of the current edit onto a space orthogonal to previous knowledge for updating. |
| Outcome: | The proposed method improves question-answering ability and hallucination mitigation by 14% and 61% for large language models after 3,000 edits. |
Think and Recall: Layer-Level Prompting for Lifelong Model Editing (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for lifelong model editing suffer from limitations in usability, such as requiring additional training corpora or lacking support for reversible and detachable edits. |
| Approach: | They propose a plug-and-play method for knowledge retrieval and storage, i.e., Layer-Level Prompting, which enables seamless and efficient lifelong model editing. |
| Outcome: | The proposed method outperforms existing methods on question answering and hallucination benchmarks across different LLMs. |